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Abstract. We investigate the differentiability properties of a real-valued qua- 
siconvex function / defined on a separable Banach space X. Continuity is only 
assumed to hold at the points of a dense subset. If so, this subset is automat- 
ically residual. 

Sample results that can be quoted without involving any new concept or 
nomenclature are as follows: (i) If / is use or strictly quasiconvex, then / 
is Hadamard differentiable at the points of a dense subset of X. (ii) If / is 
even, then / is continuous and Gateaux differentiable at the points of a dense 
subset of X. In (i) or (ii), the dense subset need not be residual but, if X is 
also reflexive, it contains the complement of a Haar null set. Furthermore, (ii) 
remains true without the evenness requirement if the definition of Gateaux 
differentiability is generalized in an unusual, but ultimately natural, way. 

The full results are much more general and substantially stronger. In par- 
ticular, they incorporate the well known theorem of Crouzeix, to the effect that 
every real-valued quasiconvex function on is Frechet differentiable a.e. 

1. Introduction 

According to Rockafellar [38l p. 428], it has been known since the early 20th 
century that a real- valued convex function on *BL N is everywhere continuous and 
a.e. Frechet differentiable, although the geometric leanings prevalent at that time 
make it difficult to pinpoint the origin of this result with greater accuracy. 

When M. N is replaced by an infinite dimensional Banach space X, a real-valued 
convex function on X is either continuous at every point or discontinuous at every 
point. This makes it obvious that the investigation of the differentiability proper- 
ties of convex functions should be limited to the former class. When X is separable, 
Mazur |31j proved the residual Gateaux differentiability of continuous convex func- 
tions in 1933. Recall that, in Baire category terminology, a residual subset is the 
complement of a set of first category. Much later, in 1976, Aronszajn [1] showed 
that Gateaux differentiability is also true almost everywhere, provided that a suit- 
able generalization of null sets is used when dim A" = oo (Aronszajn null sets; see 
Subsection 12. 1| . 

In the theorems of Mazur and Aronszajn, Gateaux differentiability can be re- 
placed by Hadamard differentiability -and hence by Frechet differentiability when 
X = M. N - since both concepts coincide for locally Lipschitz functions (this is well 
known [321 p. 19] and elementary). More generally, the residual Frechet differen- 
tiability of continuous convex functions was proved in a 1968 landmark paper by 
Asplund [2] when X* is separable, as well as in other cases that do not require or 
imply the separability of X. 

The dichotomy everywhere/nowhere continuous is no longer true for finite quasi- 
convex functions, where the quasiconvexity of / is understood as f(Xx+ (1 — X)y) < 
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max{/(x), f(y)} for every x,y and every A £ [0, 1]. Nonetheless, in 1981, Crouzeix 
[14] (see also [9]) proved that, just like convex functions, real-valued quasicon- 
vex functions on R , not necessarily continuous, are a.e. Frechet differentiable. 
To date, this property has not been extended in any form to infinite dimensional 
spaces. In fact, with only a few notable exceptions, the literature has consistently 
focused on differentiability under local Lipschitz continuity conditions, a topic with 
a long history which is still the subject of active research; see the recent text [25) . 
Of course, local Lipschitz continuity is grossly inadequate to handle quasiconvex 
functions, even continuous ones, irrespective of dim A. 

It is the purpose of this paper to show how Crouzeix's theorem can be general- 
ized when R^ is replaced by a separable Banach space A. We provide variants of 
the theorems of Mazur and Aronszajn in the more general setting of quasiconvex 
functions, although neither theorem has a genuine generalization. In particular, 
"mixed" criteria must be used to evaluate the size of the set of points of differen- 
tiability. We shall return to this and related issues further below. 

As a simple first step, recall that the set of points of discontinuity of any real- 
valued function / on a topological space X is an JF a (see e.g. [211 P-78], [3U P- 58]), 
so that either this set is of first category, or it has nonempty interioiQ. Evidently, no 
generic differentiability result should be expected in the latter case, which dictates 
confining attention to quasiconvex functions that are continuous at the points of a 
residual subset of the (separable) Banach space X. Incidentally, Mazur's theorem 
breaks down for this class of functions, even when X = R and continuity is assumed. 
Indeed, in general, a monotone continuous function on R is only differentiable at 
the points of a set of first category f [44) Corollary 1]), even though its complement 
has measure by Lebesgue's theorem. As we shall see later, Aronszajn's theorem 
also fails in the quasiconvex (even continuous) case when dim A = oo. 

In Banach spaces, residual sets are dense and the converse is trivially true for Qg 
sets. In particular, a real- valued function is continuous at the points of a residual 
subset if and only if it is densely continuous, i.e., continuous at the points of a 
dense subset. Common examples of densely continuous functions include the upper 
or lower semicontinuous functions f [321 Lemma 2.1]) and, by a theorem that goes 
back to Baire himself, the so-called functions of Baire first class, i.e., the pointwise 
limits of sequences of continuous functions ( |43[ p. 12]). An apparently new class 
(ideally quasiconvex functions) that contains all the semicontinuous quasiconvex 
functions -and even all the quasiconvex functions when X = TSL N - is described in 
Subsection 13.21 

As shown by the author in 37], when A is a Baire topological vector space (tvs), 
the densely continuous quasiconvex functions / on X have quite simple equivalent 
characterizations in terms of the lower level sets 

(1.1) F a := {x £ X : f(x) < a} and F' a := {x £ X : f(x) < a}, 

where a £ R U {— oo}. Of course, since / is real-valued, i^_oo = -F-oc = 0> but the 
notation will occasionally be convenient. By the quasiconvexity of /, all the sets 
F a and F' a are convex and, conversely, if all the F a (or F' a ) are convex, then / is 
quasiconvex, which is de Finetti's original definition [19) . 

In [37] and again in this paper, a special value plays a crucial role, which is 
the so-called topological essential infimum 7~essinfx/ of /, denoted by m for 
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simplicity, defined by 

(1.2) m := Tessinf / := sup{a £ R : F a is of first category} 

= inf{a £ K : F a is of second category}. 

The last equality in (|1.2[) follows from the fact that the sets F a are linearly ordered 
by inclusion. Since / is real-valued and X is a Baire space, it is always true that 
Tessinf^ / G [—00, 00). Note also that infx / < Tessinf^ / and that the set F m 
(but not F^) is always of first category since it is the union of countably many F a 
with a < m. 

From now on, X is a separable Banach space and / : X — > M. is quasiconvex 
and densely continuous. Below, we give a synopsis of the differentiability results 
proved in this paper. The rough principle is that the size of the subset F m ranks 
the amount of differentiability of / (the smaller the better). Even though F m is 
always of first category, various refinements will be involved. Many of the concepts 
and some technical results needed for the proofs are reviewed or introduced in the 
next two sections. In particular, several properties of F' m with direct relevance to 
the differentiability question are established in Section [3] 

In Section [4] we focus on the differentiability of / above level m, that is, at 
the points of X\F' m . The special features of F' m make it possible to rely on a 
theorem of Borwein and Wang [7] to prove that / is Hadamard differentiable on 
X\F' m except at the points of an Aronszajn null set (Theorem 14. 2j) . If m = —00, 
so that X\F{ n = X, this fully generalizes Aronszajn's theorem and settles the 
differentiability issue. 

Accordingly, in the remainder of the discussion, m > — 00 is assumed. It turns 
out that X\F! m is always semi-open, i.e., contained in the closure of its interior 
(which is not true when F' m is replaced by an arbitrary convex set), so that the 
differentiability result just mentioned above still shows that / is Hadamard differ- 
entiable at "most" points of X\F' m . That X\F' m may be empty does not invalidate 
this statement. Hence, it only remains to investigate differentiability at the points 
ofF^=F m U/- 1 (m) ; 

It turns out that / is never Gateaux differentiable at any point of F m (Theorem 
15. ip . Thus, the differentiability of / on F' m depends solely upon its differentiability 
at the points of / _1 (m) and, if / is Gateaux differentiable at x £ / _1 (m), then 
Df(x) = (Theorem 15. ip . as if m were a genuine minimum. In summary, the 
problem is to evaluate the size of those points of / _1 (m) at which the directional 
derivative of / exists and is in all directions and to decide whether / is Hadamard 
differentiable at such points. While this demonstrates the importance of m in the 
differentiability question, the task is not as simple as one might first hope. 

When X = R N , F m is not only of first category but also nowhere dense because, 
in finite dimension, convex subsets of first category are nowhere dense. Differentia- 
bility when X is separable and F m is nowhere dense is the topic of Section [5] The 
technicalities depend upon whether F' m is of first or second category but, ultimately, 
we show that / is Hadamard -but not Frechet- differentiable on the complement 
of the union of a nowhere dense set with an Aronszajn null set (Theorem 15.41) . 
Such unions cannot be replaced with Aronszajn null sets alone or with sets of first 
category alone. 

Every subset which is the union of a nowhere dense set with an Aronszajn null 
set has empty interior (of course, this is false if "nowhere dense" is replaced by "first 
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category") and the class of such subsets is an ideal, but not a (T-ideal. In particular, 
/ above -or any finite collection of similar functions- is Hadamard diffcrcntiable at 
the points of a dense subset of X. This is only a "subgeneric" differentiability 
property but, if X is separable and reflexive, a truly generic result holds: If F m is 
nowhere dense, / is Hadamard differentiable on X except at the points of a Haar 
null set (Theorem I5.5j) . This should be related to the rather unexpected fact that, 
in such spaces, a quasiconvex function is densely continuous if and only if its set of 
points of discontinuity is Haar null (Theorem 13. 3[) . 

Examples show that Theorem 15.51 is not true for Frechct differentiability, or 
if X is not reflexive, or if "Haar null" is replaced by "Aronszajn null", even if 
/ is continuous and X = i 2 . Thus, Aronszajn's theorem cannot be extended to 
(densely) continuous quasiconvex functions if dim A = oo. Nonetheless, when X = 
K. , Crouzeix's theorem is recovered because every quasiconvex function on is 
densely continuous, the Haar null sets of M. N are just the Borel subsets of Lebesgue 
measure and Hadamard and Frechet differentiability coincide on M. N . 

Aside from X — M. , there are several conditions ensuring that F m is nowhere 
dense. In particular, if / is upper semicontinuous (use) or, more generally, if 
mix f = m, for then F m = 0. Also, F m is nowhere dense if / is strongly ide- 
ally quasiconvex (Definition 13.11 Theorem 13. 4p or strictly quasiconvex and densely 
continuous (Corollary 15.61) . 

In general, F m need not be nowhere dense if dim A = oo, especially when / 
is lower semicontinuous (lsc). Thus, loosely speaking, the differentiability issue is 
more delicate for lsc functions than for use ones. However, no difficulty arises as 
a result of passing to the lsc hull, even though doing so can only enlarge F m : In 
Subsection 15.21 we show that Theorems 15.41 and 15.51 are applicable to a densely 
continuous quasiconvex function / if and only if they are applicable to its lsc hull. 

The case when F m is not nowhere dense is discussed in Section |6l This is new 
territory since it never happens when X = M. N or when / is use, let alone convex 
and continuous. We show that a subset F^ of X, often much smaller than F m , 
is intimately related to the Gateaux differentiability question. Specifically, F^ 
consists of all the limits of the convergent sequences of points of F m that remain in 
some finite dimensional subspace of X. 

In a more arcane terminology, F^ is the sequential closure of F m for the finest 
locally convex topology on X. This type of closure has been around for quite a while 
in the literature, but only in connection with issues far removed from differentia- 
bility (topology, moment problem in real algebraic geometry, etc.) and often in a 
setting that rules out infinite dimensional Banach spaces (countable dimension). 

In Theorem l6.2[ we prove that if F^ has empty interior, then / is both continuous 
and Gateaux differentiable at the points of a dense subset of X. Since Hadamard 
differentiability implies continuity, this is weaker than the analogous result when 
F m is nowhere dense (Theorem 15. 4[) , but applicable in greater generality. 

Corollaries are given in which the assumption that F^ has empty interior is re- 
placed by a more readily verifiable condition. Most notably, we show in Corollary 
16.51 that every even densely continuous quasiconvex function / on a separable Ba- 
nach space is continuous and Gateaux diffcrcntiable at the points of a dense subset 
of X. Furthermore, if X is also reflexive, / is continuous and Gateaux differentiable 
on X except at the points of a Haar null set. These properties remain true when / 
exhibits more general symmetries (Corollary I6.6|) . 
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Section [7] is devoted to an example showing that when F m is not nowhere dense, 
the differentiability properties of / at the points of F' m or, equivalently, / _1 (m), 
are indeed significantly weaker than when F m is nowhere dense. This confirms that 
an optimal outcome cannot be obtained without splitting the investigation of the 
two cases. 

The results of Section [6l particularly those incorporating some symmetry as- 
sumption about /, make it legitimate to ask whether always has empty inte- 
rior. If true, this would imply that Theorem 16.21 is always applicable. In Section 
|SJ we put an early end to this speculation, by exhibiting a convex subset C of £ 2 
of first category such that C* — i 2 . The construction of C also shows how to pro- 
duce densely continuous quasiconvex functions / such that m > — oo and F m = C, 
but we were unable to determine whether any such function fails to be Gateaux 
diffcrentiable at the points of a dense subset. 

The aftermath of F^ not always having empty interior is that, when dim A — oo, 
it remains an open question whether every densely continuous quasiconvex function 
/ on a separable Banach space X is continuous and Gateaux differentiable at the 
points of a dense subset. Nevertheless, in Section [9l this question is answered in 
the affirmative after the definition of Gateaux differentiability is slightly altered. 

A basic remark is that the concept of Gateaux derivative at a point x continues 
to make sense if it is only required that the directional derivatives at x exist and are 
represented by a continuous linear form for some residual set of directions in the unit 
sphere. This suffices to define an "essential" Gateaux derivative at a; in a unique 
way, independent of the residual set of directions (which is not true if "residual" 
is replaced by "dense"). With this extended definition, the problem is resolved in 
Theorem l9.2l In particular, if X is reflexive and separable, every densely continuous 
quasiconvex function is continuous and essentially Gateaux differentiable on X 
except at the points of a Haar null set. 

2. Preliminaries 

We collect various definitions and related results that will be used in the sequel. 
Most, but not all, of them are known, some more widely than others. Whenever 
possible, references rather than proofs are given. 

2.1. Aronszajn null sets. Let X denote a separable Banach space. If (£„) C X is 
any sequence, call 2l((£ n )) the class of all Borel subsets E C X such that E = U n E n 
where E n is a Borel "null set on every line parallel to £ n ", that is, \\{{x + R£„) 
nE n ) = for every i£X, where Ai is the one-dimensional Lebesgue measure. The 
Aronszajn null sets ([!]) are the (Borel) subsets E of X such that E 6 2l((£„)) for 
every sequence (£ n ) such that span{(£„)} = X. 

The Aronszajn null sets form a cr-ring that does not contain any nonempty open 
subset and every Borel subset of an Aronszajn null set is Aronszajn null. When 
X = R N , they are the Borel subsets of Lebesgue measure 0. It was shown by 
Csornyei |16) that the Aronszajn null sets coincide with the Gaussian null sets 
(Phelps [34]; these are the Borel subsets of X that are null for every Gaussian^ 
probability measure on A) and also with the "cube null" sets of Mankiewicz |26j . 



2 For every t £ X* , the measure fi e on R defined by [i t (S) := fl(£ has a Gaussian 

distribution. 
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Aronszajn null sets are preserved by affine diffcomorphisms of X, i.e., translations 
and bounded invertible linear transformations. As noted by Matouskova [28], if C 
is a convex subset of X with nonempty interior, the boundary dC is Aronszajn 

o 

null, but this need not be true if C = 0, even if C is closed and X is Hilbcrt. 

2.2. Haar null sets. Let once again X denote a separable Banach space. The 
Borel subset E c X is said to be Haar null if there is a Borel probability measure 
fj, on X such that fi(x+E) = for every x £ X. This definition is due to Christensen 
[TO] . [IT]; see also [29]. It is obvious that every Aronszajn null set is Haar null, but 
the converse is false [31], except when X = Mr. The Haar null sets also form a cr-ring 
(though this is not obvious from just the definition) containing no nonempty open 
subset, every Borel subset of a Haar null set is Haar null and, just like Aronszajn 
null sets, they are preserved by affine diffeomorphisms. 

In practice, it is not essential that fi above be a probability measure. The 
definition is unchanged if fi is a Borel measure having the same null sets as a 
probability measure. For example, this happens if /i(E) := Ai(R£ fl E) where 
£ G A\{0} and Ai is the Lebesgue measure on R£ (change Ai into <p\i with ip(t) := 
7T 2 e 1 ). 

If, in addition, X is reflexive, every closed convex subset of X with empty in- 
terior is Haar null (Matouskova [30.) but, as noted in the previous subsection, not 
necessarily Aronszajn null. 

2.3. Gateaux and Hadamard differentiability. A real-valued function / on 
an open subset V of a Banach space X is Gateaux differentiable at x £ V if there 
is l x G X* such that l x (h) = lim f ^o ^ x+th ^~^ x ^ for every h G X. If so, l x is 
denoted by Df(x). If, in addition, the limit is uniform for h in every compact 
subset of X, then / is said to be Hadamard differentiable at x. Equivalently, / is 
Hadamard differentiable at x if and only if Df(x)h — lim tii _>o f( x+tn hn)-f(x) f or 
every sequence (t n ) C R\{0} tending to and every sequence (h n ) C X tending 
to h in X. If / is Hadamard differentiable at x, then it is continuous at x. This 
is folklore (see e.g. [3]) and elementary. Of course, Gateaux differentiability alone 
does not ensure continuity. When X = K. N , it is plain from the first definition that 
Hadamard differentiability and Frechet differentiability coincide. 

2.4. Cone monotone functions. Let X be a Banach space and K C X be a 
closed convex cone with nonempty interior. If V C X is a nonempty open subset, 
a function / : V — > M is said to be K -nondecreasing if x £ V, k € K\{0} and 
x + k G V imply f(x) < f(x + k). This concept has long proved adequate to 
extend Lebesgue's theorem on differentiation of monotone functions. It goes back 
(in R 2 ) to the 1937 edition of the book by Saks [3D] and resurfaced in the work of 
Chabrillac and Crouzeix 9 . Its use in separable Banach spaces, by Borwein et al. 
[5] and Borwein and Wang [7], is more recent. The following result is essentially [3 
Theorem 18]. 

Theorem 2.1. (Borwein- Wang) . Let X be a separable Banach space, K C X be 

o 

a closed convex cone with K ^ and V C X be a nonempty open subset. Suppose 
that f : V — R is K -nondecreasing. Then f is Hadamard differentiable on V except 
at the points of an Aronszajn null set. 
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In [7], Theorem 12.11 is proved only when V — X but, as shown below, it is not 
difficult to obtain the general case as a corollary. If x, y € X, we use the notation 
y <k x if x — y E K and y <k x if x — y £ i4T\{0}. These relations are obviously 
transitive since X is stable under addition (but <k is an ordering if and only if 
K H (—if) = {0}; this is unimportant). 

First, / in Theorem 1 2. II is locally bounded (above and below) on V. To see this, 

o 

let x E V be given and choose k E K. In particular, fc ^ 0. After replacing A; by tk 

o 

for f > small enough, which does not affect fc € if, it follows from the openness 
of V that we may assume that x — k E V and x + k EV. If z £ X and | \z — x\ \ < e 

o 

with e > small enough, then z E V and k + (z — x) 6 if, so that x — k <k z, and 

o 

k — (z — x) E K 1 so that z <k x + k. Therefore, f(x — k) < f{z) < f(x + k), which 
proves the claim. 

From the above, for every x E V, there is an open neighborhood U x C V of x 
such that / is bounded on U x and, by the separability of X, there is a covering 
of V by countably many U x . Thus, since a countable union of Aronszajn null sets 
is Aronszajn null and since U x C V implies that / is if-nondecreasing on U x , it 
is not restrictive to prove Theorem 12.11 with V replaced by U x or, equivalently, to 
prove it under the additional assumption that / is bounded above and below on 
V. This can be done by using Theorem 12.11 with V = X and / replaced by a finite 
if-nondecreasing extension of / to X. Such an extension can be obtained as follows: 
Since / is bounded on V, let a, b E R be such that a < f(x) < b for every x E V 
and set 

j, s _ [ a if x £ V + K, 

^ X >-\ sup{/(y) : y E V, y < K x} if x E V + K. 

Since (V + K) + K = V + K and <k is transitive, it is straightforward to check 
that / is if-nondecreasing on X, that / = / on V and that a < f < b, so that / 
is real-valued (when x E V + K , the set {y E V, y <k x} is not empty, so that the 
supremum in the definition of / is never — oo). Other extensions are described in 
[7], but they need not be finite when / is finite. 

Remark 2.1. The Borwein-Wang theorem was refined by Duda |18[ Remark 5.2]: 
/ is Hadamard differentiable on V except at the points of a subset in a class C 
introduced by Preiss and Zajicek, which is strictly smaller than the class of Aron- 
szajn null sets Proposition 13]. The class C may replace the Aronszajn null 
sets everywhere in this paper. 

2.5. Convex sets and category. Convex sets have a few special properties rela- 
tive to Baire category. Two useful ones are given below. Like several other results, 
they are stated in Banach spaces but are valid in greater generality. 

Lemma 2.2. (37, Lemma 3.1]) Let X be a Banach space and let U and G be subsets 
of X with U open and G convex. If A C X is of first category and U\A C G, then 
UcG. 

The next lemma is a by-product of Lemma 12.21 

Lemma 2.3. fl37|, Lemma 3.2]) If X is a Banach space and C £ X is convex, 
then X\C is locally of Baire second category in X, that is, for every open subset 
U C X such that U n (X\C) is nonempty, U D (X\C) is of second category in X. 
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Of course, the spirit of Lemma \2. 31 is that even locally, the exterior of a convex 
subset C ^ X is always a large set in some sense. While intuitively clear when 
X = M. N , the existence of dense and convex proper subsets of infinite dimensional 
Banach spaces makes this issue less transparent in general. 

3. Densely continuous quasiconvex functions 

In the Introduction, we defined the densely continuous real valued functions on 
a tvs X to be those functions that are continuous at the points of a dense subset 
of X. Recall that when X is a Baire space, for instance a Banach space, this is 
actually equivalent to continuity at the points of a residual subset of X. 

3.1. Some general properties. The first theorem of this section is part of the 
main result of [37] . It gives a characterization of the densely continuous quasiconvex 
functions in terms of their lower level sets. The notation ()l.lj) is used and will be 
used throughout the paper. 

Theorem 3.1. / |37[ Corollary 5.3]) Let X be a Banach space and let f : X —> K 

be quasiconvex. Set m := Tessinfx / (see Hl.ty) ). Then, f is densely continuous if 

o 

and only if (i) F a ^ when a > m and (ii) either m — — oo or m > — oo and F a 
is nowhere dense when a < m. 

Other conditions equivalent to, or implying, dense continuity for quasiconvex 
functions are given in [37) . but Theorem 13.11 will suffice for our purposes. Since 
"first category" , "nowhere dense" and "empty interior" are synonymous for convex 
subsets of R w , Theorem 13.11 shows that every real- valued quasiconvex function on 
R N is densely continuous. This argument is independent of Crouzeix's theorem 
mentioned in the Introduction. 

In the next theorem, we prove a number of properties of the set F' m which will be 
instrumental in the discussion of the differentiability question. Part (i) was already 
noticed in [37] . The very short proof is given for convenience. 

Theorem 3.2. Let X be a Banach space and let f : X — > R be quasiconvex and 
densely continuous. Set m := Tessinfx /• 

(i) If F' m (i.e., f~ l {m)) is of first category, then F' m (and hence also f (m)) is 
nowhere dense. 

o 

(ii) If ' F' m (i.e., f~ l {m)) is of second category, then F^ ^ 0. In particular, dF' m = 

o 

dF' m is nowhere dense. If X is separable, dF' m is Aronszajn null. 

(Hi) X\F{ 11 is semi-open, i.e., contained in the closure of its interior X\F m . 

(iv) If X is separable, the set of points of discontinuity of f in X\F} n is contained 
in an Aronszajn null set. 

(v) If X is separable, F' m is contained in an Aronszajn null (Haar null) set if and 
only if F m is Aronszajn null (Haar null). 

Proof, (i) If x G F m \F' n , then / is not continuous at x, so that F m C F' m U A 
where A is the set of points of discontinuity of /. Since / is densely continuous, A 
is of first category, whence F m is of first category, that is, F' m is nowhere dense. 

o 

(ii) Since F' m is of second category, F m ^ 0. On the other hand, if x G F m and 
/ is continuous at x, then x £ F' m . Since the set A of points of discontinuity of / is 
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of first category, this means that F m \A C F' m . In particular, F m \A C F' m so that, 

_° -/ ° -°, 

by Lemmas F m C F^. Thus, F m C F/„ and so F' m = F m ^ 0. 

A convex set with nonempty interior has the same boundary as its closure ([5J 

p. 105]) and the boundary of a closed set is nowhere dense. Thus, dF' m is nowhere 

o 

dense. Since F' m ^ and F' m is convex, dF' m is Aronszajn null when X is separable 
(Subsection uTT]) . 

(iii) When dim A = oo, this does not follow from the convexity of F' m alone (the 
complement of a dense convex subset is not semi-open) but, by (i) and (ii), we also 

o 

know that either F' m is nowhere dense, or F' m ^ 0. 

If F' m is nowhere dense, X\F m is dense in X and therefore in X\F} n . Suppose 
° _, 
now that F' m ^ 0. The only points of X\F^ which are not already in X\F m lie 

on d(X\Fl n ) = dF^. As just pointed out in the proof of (ii), F' m and F m have the 

o 

same boundary because F' m is convex and F' m ^ 0. Therefore, if x € X\F' n and 
x X\F m , then x e dF m = d(X\F m ) is in the closure of X\F m . 

(iv) As before, let A denote the set of points of discontinuity of /. The claim is 
that A\F{ n is contained in an Aronszajn null set. If £ £ A\F} n , then f{x) > m and 
there are a sequence (x n ) C X with lirrin-^oo x n = x and a £ R such that either 
f{x n ) < a < f(x) for every n £ N or f(x) < a < f(x n ) for every n£l. With no 
loss of generality, we may assume that a£Q and, since f{x) > m, that a > m (of 
course, this is redundant when f(x) < a). 

In the first case (i.e., f(x n ) < a < f(x)), x € F a but x ^ F a , so that x G dF a . 

o 

In the second (i.e., f(x) < a < f(x n )), x g F a but x £ F a , so that once again 

o 

x G dF a . Thus, A\Fj n C U a eQ,a>mdF a . By Theorem 13. 1\ F a ^% when a > to, so 
that dF a is Aronszajn null since F a is convex (Subsection l2.1[) and so U a eQ,a>mdF a 
is Aronszajn null. 

(v) The sufficiency is trivial. To prove the necessity, suppose then that F' m 
is contained in an Aronszajn null (Haar null) set. As noted in the proof of (i) 

above, F' m C F' m U A where A is the set of points of discontinuity of / and so 
F m C F' m U (A\Fj n ). By (iv), A\F^ is contained in an Aronszajn null set, so that 
F m is Aronszajn null (Haar null) since it is Borel. □ 

While the quasiconvexity of / is actually unnecessary in part (i) of Theorem [321 
it is important in part (ii). By Theorem l3.il every lower level set F' a -not just F' m - 
is either nowhere dense or has nonempty interior, but it is only when a = to that 
both options are possible (F' a is nowhere dense when a < m since F' a C Fp with 
a < ft < to and has nonempty interior when a > m). In contrast, F m is always of 
first category but it may or may not be nowhere dense. 

Remark 3.1. It is readily checked that the complement of an Aronszajn null set in 
a semi-open subset is dense in that subset. Thus, when X is separable, parts (iii) 
and (iv) of Theorem \3.2\ show that the set of points of continuity of f in X\Fl m is 
dense in X\F! m . 

In a separable and reflexive Banach space, a finite quasiconvex function is densely 
continuous if and only if its set of points of discontinuity is Haar null. This perhaps 
surprising result is proved next. 
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Theorem 3.3. Let X be a reflexive and separable Banach space. The quasicon- 
vex function f : X — > K is densely continuous if and only if the set of points of 
discontinuity of f is Haar null. 

Proof. Since the complement of a Haar null set is dense, the "if" part is clear. 
Conversely, since the set A of points of discontinuity of / is an T a (hence Borel), 
it suffices to show that if / is densely continuous, A is contained in a Haar null set. 
This will be seen by a suitable modification of the proof of part (iv) of Theorem 

If x G A, there are a sequence [x n ) C X with lim„_>. 00 x n — x and a G R such that 
either f(x n ) < a < f(x) for every neNor f(x) < a < f(x n ) for every neN. With 
no loss of generality, we may assume that a € Q and a=£ m := Tessinfx / < oo. 

In the first G F a but x F a , so that x G dF a and, in the second, x G F a 

o 

but x F a , so that once again x G dF a . If (m > — oo and) a < to, it follows 
from Theorem 13.11 that dF a — F a has empty interior. Since X is reflexive and 

o 

separable, dF a is Haar null (Subsection 12. 2\i . If a > m, then F a ^ by Theorem 
13.11 so that dF a is Aronszajn null (Subsection 12. 1[) and therefore Haar null. Thus, 
U a e®\{m}dF a is Haar null and A C U a £Q\{ m ydF a . Since A is Borel -indeed an T a - 
it is Haar null. □ 

When X — R N , the set of points of discontinuity of / is even er-porous; see 
Theorem 19] (as pointed out in [37l Remark 5.2], the lsc assumption in that 
theorem is not needed). 

3.2. Ideally quasiconvex functions. The subset C of the Banach space X is 
said to be ideally convex if Yi^ =l \ n x n G C for every bounded sequence (x n ) G C 
and every sequence (A„) G [0, 1] such that Y^ =1 X n = 1. The boundedness of (x n ) 
ensures that T,^ =1 X n x n is absolutely convergent. Such subsets were introduced by 
Lifsic 24 in 1970 but, apparently, they have not been used in connection with 
quasiconvex functions. Indeed, without Theorem 13.11 the purpose of doing so is 
not apparent. 

Definition 3.1. The function f : X — >• R is ideally quasiconvex (strongly ideally 
quasiconvex) if its lower level sets F' a (F a ) are ideally convex. 

It is readily seen that a strongly ideally quasiconvex function is ideally quasicon- 
vex (use F' a = CipyaFp), but the converse is false; see Remark [731 later. Also, / 
is ideally quasiconvex if and only if / (S^L X \ n x n ) < sup„ f{x n ) for every bounded 
sequence (x n ) C X and every sequence (A n ) C [0, 1] such that E^L 1 A„ = 1. Strong 
ideal convexity amounts to / (T,^ =1 X n x n ) < a whenever f(x n ) < a for every n. This 
has nothing to do with strict quasiconvexity. For instance, every use quasiconvex 
function is strongly ideally quasiconvex (see below). 

If C G X is ideally convex, it is CS-closed as defined by Jameson [22] and the 
two concepts coincide when C is bounded. By a straightforward generalization 
of Caratheodory's theorem (see for example [13]), every convex subset of R^ is 
ideally convex. Therefore, every quasiconvex function on R N is strongly ideally 
quasiconvex. Open convex subsets are CS-closed ([20 ), hence ideally convex, and 
the same thing is trivially true of closed convex subsets. In particular, lsc (use) 
quasiconvex functions are ideally quasiconvex (strongly ideally quasiconvex). 

We shall also need the remark that an ideally convex subset C can only be 
nowhere dense or have nonempty interior. Indeed, if C is not nowhere dense, CC\B 
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is dense in B for some nonempty open ball B. Since C and B are ideally convex, 
C fl B is ideally convex (obvious) and bounded and therefore CS-closed. By [22l 
Corollary 1], this implies that C fl B = B, so that B C C and so C has nonempty 
interior. 

The main properties of (strongly) ideally quasiconvex functions are captured in 
the next theorem. 

Theorem 3.4. Let X be a Banach space. 

(i) If f : X R is ideally quasiconvex, then f is quasiconvex and densely continu- 
ous. 

(ii) If f : X — > R is strongly ideally quasiconvex and m := TesshrPjf /, then f is 
quasiconvex and densely continuous and F m is nowhere dense. More generally, this 
is true if f is ideally quasiconvex and F m is ideally convex. 

Proof, (i) If a > m, then F' a is ideally convex but not nowhere dense since F' a D F a 

o 

is of second category by definition of m. Thus F' a ^ by the remark before the 
theorem. Now, let a < m and, by contradiction, suppose that F a is not nowhere 
dense, so that the larger F' a (ideally convex) is not nowhere dense either. By the 

o o 

remark before the theorem, F' a ^ and so Fp ^ for every j3 > a. But Fp is of 
first category by definition of m if (3 G (a,m) and a contradiction arises. Thus, 
F a is nowhere dense when a < m. That / is densely continuous now follows from 
Theorem 13.11 

(ii) By (i), / is densely continuous. Since F m is ideally convex and has empty 
interior (it is of first category), it is nowhere dense, once again by the remark before 
the theorem. □ 

Strongly ideally quasiconvex functions are the closest generalization of finite 
dimensional quasiconvex functions. As we shall see in Section [5l both have the 
same differentiability properties. By Theorem 13. 41 (i). ideally quasiconvex functions 
possess only the continuity properties of the finite dimensional case. 

4. Differentiability above the essential infimum 

Throughout this section, X is a separable Banach space and / : X — > R is 
quasiconvex and densely continuous. With m := T ess mix f, the goal is to prove 
that / is Hadamard diffcrentiable at every point of X\F! m except at the points of 
an Aronszajn null set. 

The last preliminary lemma will enable us to use Theorem 12.11 to settle the 
differentiability question at points of X\F! m . The line of argument of the proof, but 
with other assumptions, has been used before (p~5j Theorem 3.1], Proposition 
2])- 

Lemma 4.1. Let x G X\F{ n be a point of continuity of f. Then, x 6 X\F m and 
there is an open neighborhood U x of x contained in X\F m and a closed convex cone 
K x C X with nonempty interior such that f is K x -nondecreasing on U x - 

o 

Proof. Since f(x) > m, choose a s (m, f(x)), so that F a =^ by Theorem l3.ll Pick 

o 

x- e -F Q and e > small enough that B{x~ , 2e) C F a , that U x := B (x, e) C X\F' a 
(this is possible since f(x) > a and / is continuous at x; in particular, U x is 



12 



PATRICK J. RABIER 



contained in the interior X\F' m of X\F{ n ) and that ^ B(2(x — x_), 2s). Now, set 
x + := 2x — X-, so that x is the midpoint of x_ and x + and 

K x :=Ux> \B(x + -a:_,2e) = U x > Q \B(2(x - x_),2e). 

Clearly, K x is a convex cone with nonempty interior. That it is closed easily 
follows from the assumption ^ B(2(x — x~),2e). Also, if k G K x \{0}, then 
x — G B (x-,e) where A > is chosen such that k G \B{x+ — X-, 2e). Hence, if 
y G U x — B (x, e) , it follows that y - ^ = (y - x) + (x - ^) G B(x- , 2s). 

Suppose now that y G U x ,k <E K x \{0} and y + fc £ J/ x . From the above, y — jx = 
z G B(x^,2e). By writing ^_ — _1_ f or some t € (0, 1), this yields y — t(y + k) + 
(1 — t)z. Since / is quasiconvex, f{y) < max{/(j/ + k), f(z)} — f(y + k), the latter 
because f(z) < a < f(y + k) (recall z G F a and y + k E U x C A\F^). □ 

Theorem 4.2. Let X be a separable Banach space and let f : X — » R fee gita- 
siconvex and densely continuous. Set m :— 7"essinfx/- Then, f is Hadamard 
differentiable on X\F[ n except at the points of an Aronszajn null set. 

Proof. With no loss of generality, assume that X\F' n ^= 0. Since X\F m is dense 
in X\F^ (Theorem (hi)), it follows that X\F' m ^ 0. If C m denotes the set of 
points of continuity of / in X\F^ then C m C X\F m by Lemma H. II and C m ^ 
since X\F m is open and nonempty and / is densely continuous. 

If x G C m , it follows from Lemma [4.11 and Theorem 12.11 that there is an open 
neighborhood U x of x contained in X\F m such that / is Hadamard differentiable 
on U x except at the points of an Aronszajn null set. Since X is separable, the open 
set U :— U x ec m U x C X\F m is Lindelof and therefore coincides with the union of 
countably many U x . As a result, / is Hadamard differentiable on U except at the 
points of an Aronszajn null set. 

Lastly, since U contains all the points of continuity of / in X\F' m , the points of 
(A\F^)\(7 are the points of discontinuity of / in X\F^. By Theorem l3.2l fiv). these 
points are contained in an Aronszajn null set, so that, as claimed, / is Hadamard 
differentiable on X\F! m except at the points of an Aronszajn null set. □ 

A variant of Remark 1 3 . 1 1 may be repeated: Since X\F! m is semi-open (Theorem 
13.21 (hi)), Theorem 14.21 implies that / is Hadamard differentiable at the points of 
a dense subset of X\F^ n . The next corollary is obvious and settles the case when 
m = — oo. 

Corollary 4.3. Let X be a separable Banach space and let f : X — > R be quasi- 
convex and densely continuous. If 1~ essmfx f — — oo, then f is Hadamard differ- 
entiable on X except at the points of an Aronszajn null set. 

5. Hadamard differentiability when F m is nowhere dense 

Since the case when m := Tessinf^ / = — oo was settled in Corollary 14. 3[ it is 
henceforth assumed that m > — oo. To avoid giving the impression that this is a 
restrictive assumption, this is not recorded in the statement of the results, which are 
all true when m = — oo (by Corollary 14.31 or because their assumptions implicitly 
rule out m = — oo). 

Theorem 14.21 answers the differentiability question at points of A\F^ t , but it 
remains to investigate the differentiability at points of F' m . To begin with, we show 
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that if x £ F' n , the Gateaux derivative Df(x) can only exist if x £ f (m) and, if 
so, that Df(x) = 0. In particular, this shows that when it comes to differentiability, 
m has the same property as a pointwise infimum. 

Theorem 5.1. Let X be a Banach space and let f : X ^ R be quasiconvex. 

(i) If x £ F m , then f is not Gateaux differentiable at x. 

(ii) If x £ f (m) and f is Gateaux differentiable at x, the Gateaux derivative 
Df(x) is 0. 

Proof, (i) Since Gateaux differentiability does not imply continuity, this does not 
follow from the fact that / is not continuous at any point of F m ([371 Remark 4.1]). 

With no loss of generality, assume that x = £ F m . By the convexity of F m , 
the cone CF m := U> > oAi 7 ' r „ is the countable union CF rn = U n ^nF m and hence of 
first category. Thus, X\CF m is nonempty and, if h £ X\CF m , then h ^ (since 
£ F m C CF m ) and th ^ F m for every t > 0. As a result, f(th) > m when t > 0. 
Since /(0) < m, it follows that lim t _,.o+ ^ th ^~^^ = oo, so that / is not Gateaux 
differentiable at 0. 

(ii) Once again, by translation, we may assume that x = (but now ^ F m 
since /(0) = m). By contradiction, assume that Df(0) ^ 0, so that E := {h £ X : 
Df(0)h < 0} is an open half-space. If h £ E, then f(th) < /(0) = m for t > small 
enough, say < t < t h . Thus, G := {th : h £ S, < t < t h } C F m and so G is of 
first category. On the other hand, if h £ S, then n" 1 /! 6 G if n G N and n" 1 < </j. 
In other words, E C U„ £ n nG. Since G is of first category, E is of first category. 
Therefore, it cannot be an open half-space and a contradiction is reached. □ 

The proof of part (ii) of Theorem 15.11 does not use the quasiconvexity of /. At 
any rate, Theorem 15.11 makes the differentiability of / on F/ n look like a rather 
straightforward problem: It suffices to check whether the directional derivative of 
/ at x £ / _1 (m) exists and is in every direction. However, the evaluation of the 
size of the set of such points is not that simple. The remainder of this section deals 
with the case when F m is nowhere dense. 

If F' m is of first category, then F' m is nowhere dense (Theorem 13. 21 (i)). In partic- 
ular, F m is nowhere dense and there is no need to investigate the differentiability 
of / at the points of /~ 1 (m) to prove the "subgeneric" Hadamard differentiability 
of/: 

Theorem 5.2. Let X be a separable Banach space and let f : X — >• R be qua- 
siconvex and densely continuous. With m := Tessinfx/, suppose that F' m (i.e., 
f~ 1 {m)) is of first category. Then, F' m is nowhere dense and f is Hadamard differ- 
entiable on the open and dense subset X\F m except at the points of an Aronszajn 
null set. 

In particular, if F' m is contained in an Aronszajn null (Haar null) set, then f is 
Hadamard differentiable on X except at the points of an Aronszajn null (Haar null) 
set. 

Proof. As just recalled above, F' m is nowhere dense. Thus, X\F' m is open and 
dense in X and it suffices to use Theorem 14.21 

If F' m is contained in an Aronszajn null (Haar null) set, then F m is Aronszajn 
null (Haar null) by Theorem 13.21 (v) and so it has empty interior. Thus, F' m is of 
first category and the first part applies. That X\F' m may be replaced by X in the 
first part is obvious. □ 
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We now prove a similar result when F' m is of second category. 

Theorem 5.3. Let X be a separable Banach space and let f : X — s> K be qua- 
siconvex and densely continuous. With m := 7~essinfx/, suppose that F' m (i.e., 
f~ 1 {m)) is of second category and that F m is nowhere dense. Then, f is Hadamard 
differentiable on the open and dense subset X\F m except at the points of an Aron- 
szajn null set. 

In particular, if F m is Aronszajn null (Haar null), then f is Hadamard differen- 
tiable on X, except at the points of an Aronszajn null (Haar null) set. 

o 

Proof. By Theorem 13.21 (ii) , F' rn ^ and dF' rn is Aronszajn null. On the other 

o o o 

hand, since F' m ^ and F m is nowhere dense, F m \F m is open and dense in F' m . 

o 

Furthermore, since / = m on F m \F m and hence on F' n \F m , it follows that / 

o 

is Frechet differentiable with derivative on F m \F m . Accordingly, the points of 
F} n \F m where / is not Hadamard differentiable are contained in the Aronszajn 
null set dF' m . By Theorem 14.21 the points of X\F! m where / is not Hadamard 
differentiable are also contained in an Aronszajn null set and the conclusion follows 
from X\F m c (X\F m ) U (F m \F m ). 

If F m is Aronszajn null (Haar null), it has empty interior, so that F m is nowhere 
dense and the first part applies. That X\F m may be replaced by X in the first 
part is obvious. □ 

By combining Theorem 15.21 and Theorem 15. 3[ we find 

Theorem 5.4. Let X be a separable Banach space and let f : X — > R be quasi- 
convex and densely continuous. With that m := 7"essinfx/, suppose that F m is 
nowhere dense. Then, f is Hadamard differentiable on an open and dense subset 
of X except at the points of an Aronszajn null set. 

In particular, if F' m is contained in an Aronszajn null (Haar null) set, or if F' m 
is of second category and F m is Aronszajn null (Haar null), then f is Hadamard 
differentiable on X except at the points of an Aronszajn null (Haar null) set. 

The union of two subsets of A, one of which is nowhere dense -not just of 
first category- and the other Aronszajn null, has empty interior (its complement 
is dense). Every finite -but not countable- union of such sets is of the same type. 
Thus, Theorem 15.41 not only ensures that / is Hadamard differentiable on a dense 
subset of X but also that finitely many functions satisfying its hypotheses are 
simultaneously Hadamard differentiable on the same dense subset of X. Since this 
need not be true for countably many functions, this property can only be called 
subgeneric (but see Theorem 15.51 below) . 

Remark 5.1. Phelps' example 35, p. 80] of a Gateaux differentiable norm on 
I 1 which is nowhere Frechet differentiable shows that Hadamard differentiability 
cannot be replaced by Frechet differentiability in Theorem \5.4\ 

If / is convex and continuous, it follows from [I] that / is Hadamard differentiable 
except at the points of an Aronszajn null set. By Theorem 15.41 this is still true 
when / is only quasiconvex and continuous if F' m is of second category (because 
F m = 0), but false if F' m (closed) is of first category and not Aronszajn null. For 
instance, [6j Example 6] is a counter-example with X Hilbert. However, when X is 
reflexive, Theorem 15.41 still yields a full generalization of Crouzeix's theorem: 
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Theorem 5.5. Let X be a reflexive and separable Banach space and let f : X — > R 
be quasiconvex and densely continuous. With m := T essinfx /, suppose that F m 
is nowhere dense. Then, f is Hadamard differentiate on X except at the points of 
a Haar null set. 

Proof. Recall that in reflexive separable Banach spaces, closed convex subsets with 
empty interior are Haar null ( Subsection I2.2|) . In particular, since F m is nowhere 
dense, F m is Haar null. If F' m is of second category, the result follows directly from 
Theorem 15.41 On the other hand, if F' m is of first category, then F' m is actually 
nowhere dense by Theorem l3.2l (i). Thus, F m is Haar null and Theorem 15 .41 is once 
again applicable. □ 

Theorem 15.51 should be put in the perspective of Theorem 13.31 When X = M. N , 
every quasiconvex function is densely continuous, F m is always nowhere dense and 
Haar null sets have Lebesgue measure 0, so that Crouzeix's theorem is recovered. 

Theorem 15.51 is not true if X is not reflexive: Every nonreflexive Banach space 
contains a closed convex subset C with empty interior which is not Haar null ([28]). 
If / : — Xx\Gi then / is quasiconvex and continuous on X\C (open and dense in 
X) and hence densely continuous. Also, m = 1 and F\ = C. Thus, / is not even 
Gateaux differentiable at any point of C by Theorem 15.11 

Remark 5.2. Just like Theorem \5.4\ Theorem ] 5. 5\ alreadu fails in the convex case 
if Hadamard differentiability is replaced by Frechet differentiability: In |27j . an 
example is given of an equivalent norm on £ 2 which is Frechet differentiable only 
at the points of an Aronszajn null set. Obviously, no such set is the complement of 
a Haar null set. 

5.1. Special cases. There are a number of special cases of Theorem 15 .41 and The- 
orem 15.51 In particular, by Theorem 13.41 they are always applicable when / is 
strongly ideally quasiconvex (e.g., use) or just ideally convex with F m ideally con- 
vex. More generally, the dense continuity assumption is satisfied if / is ideally 
convex (Theorem 13. 4p or, as pointed out in the Introduction, if / is the pointwise 
limit of continuous functions. Dense continuity can also be deduced from Theorem 
13. II or other criteria in [37]. 




The condition that F m is nowhere dense holds trivially if infx / = m (in 
particular, if m = —oo), for then F m = 0. This happens when / is use, but 
in other cases as well; see for instance Theorem 15.71 As shown below, F m is 
nowhere dense -and more- when / is strictly quasiconvex. Recall that this means 
f{Xx + (1 — X)y) < max{/(x), f(y)} whenever x ^ y and A £ (0,1). It is not 
hard to check that / is strictly quasiconvex if and only if it is quasiconvex and 
f{Xx + (1 — X)y) < f(x) whenever x ^ y, f(x) = f{y) and A £ (0, 1). This is also 
equivalent to saying that / is quasiconvex and nonconstant on any nontrivial line 
segment ([TTJ Theorem 9]). 

Corollary 5.6. Let X be a separable Banach space and let f : X — > R be strictly 
quasiconvex and densely continuous. Set m :— 7~ ess inf x Then, F' m is nowhere 
dense and f is Hadamard differentiable on a dense open subset of X except at the 
points of an Aronszajn null set. Furthermore: 

(i) If ' F' m is contained in an Aronszajn null (Haar null) set, then f is Hadamard 
differentiable on X except at the points of an Aronszajn null (Haar null) set. 




1(5 



PATRICK J. RABIER 



(ii) If X is reflexive, f is Hadamard differentiable on X except at the points of a 
Haar null set. 

Proof. By Theorem 13.21 (i). F' m is nowhere dense if (and only if) /~ 1 (m) is of first 
category. If so, F m C F' m is also nowhere dense, so that everything follows once 
again from Theorem 15.41 and Theorem 15.51 

To complete the proof, we show that / _1 (m) is of first category. If F m — 0, the 
strict quasiconvexity of / shows that / (m) — F' m contains at most one point. 
If F m 7^ 0, assume with no loss of generality that G F m . If x G / _1 (m) then 
/ {kx\ < max{/(0), f(x)} — m and so \x G F m . As a result, / _1 (m) C 2F m . Since 
F m is of first category, the same thing is true of 2F m and hence of f~ 1 (m). □ 

From the above proof, Corollary 15.61 is still true if f\pi is strictly quasiconvex. 
Unlike ideal quasiconvexity, strict quasiconvexity does not imply dense continuity 
and must be checked separately. (The sum of a continuous strictly convex function 
and a real-valued nowhere continuous convex function is strictly convex, hence 
strictly quasiconvex, but nowhere continuous.) 

We now show that (sub)generic Hadamard differentiability can always be ob- 
tained after a rather mild modification of the function / that does not affect the 
Hadamard derivative of / at any point where such a derivative exists. 

Theorem 5.7. Let X be a separable Banach space and let f : X — s> K be qua- 
siconvex and densely continuous. Suppose that m := 7~essinfx/ > — oo and set 
g := max{/, m}. The following properties hold: 

(i) g is quasiconvex and densely continuous and f — g in the residual set X\F m . 

(ii) If f is Hadamard differentiable at x, then g is Hadamard differentiable at x and 
Df(x)=Dg(x). 

(Hi) g is Hadamard differentiable on a dense open subset of X except at the points 
of an Aronszajn null set. Furthermore: 

(iii-l) If F' m is of first category and contained in an Aronszajn null (Haar null) set, 
or if F^ n is of second category, then g is Hadamard differentiable on X except at 
the points of an Aronszajn null (Haar null) set. 

(Hi- 2) If X is reflexive, g is Hadamard differentiable on X except at the points of 
a Haar null set. 

Proof, (i) is obvious since every point of continuity of / is a point of continuity of 
g. To prove (ii), assume that / is Hadamard differentiable at x. If x ^ F^, then 
/ is continuous at x and so / and g coincide on a neighborhood of x. Therefore, 
g is Hadamard differentiable at x and Dg{x) — Df(x). If x G F' m , it follows from 
Theorem O that f{x) = m and that Df(x) = 0. Thus, lim t _, /(x+tfe t W(x) = 
uniformly for h in the compact subsets of X. Also, g{x) = m and g(x + th) = 
f(x + th) if x + th 6 F' m , whereas g(x + th) = m if x + th £ F m . As a result, 

\g(x+th)-g(x)\ < \f(x+th)-f(x)\ ^ gQ g(x+th)-g(x) = q uniformly for h m the 

compact subsets of X. This shows that g is Hadamard differentiable at x and that 
Dg{x) = 0. 

(iii) Clearly, Tessinfx g = m, G m := {x G X : g(x) < m} — and G' m := {x G 
X : g(x) < m} — F' m , so that it suffices to use Theorem 15.41 and Theorem 15 . 5 1 with 
/ replaced by g. □ 

5.2. Differentiability of lsc hulls. Earlier, we established that every use qua- 
siconvex function / is Hadamard differentiable on a large subset because (a) / is 
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densely continuous and (b) F m = 0. In contrast, if / is lsc, (a) is still true but 
(b) holds only if mix f = m. Worse, F m need not be nowhere dense (see Remark 
17. ip . However, as we shall see, this cannot happen by just passing from a function 
/ to its lsc hull / (largest lsc function majorized by /). More specifically, if / is 

quasiconvex and densely continuous, we shall see that Theorem 15.41 and Theorem 
15.51 are simultaneously applicable to / and /. 

Recall that f(x) := limmfj/^j, f{ x ') = inf{a el:i£ F a }. Either characteri- 
zation shows that if / is quasiconvex, then / is quasiconvex, but / may achieve the 
value — oo. Accordingly, x E X will be called a point of continuity of / if f(x) E K 
and / is continuous at x. Equivalently, a point of discontinuity x of / is a point 
where f(x) = — oo, or f(x) E K but / is not continuous at x. 

Lemma 5.8. Let f : X —> K be quasiconvex and densely continuous and let f 
denote its lsc hull. Set m := 7~essinfj5f / and G m '■= {x £ X : f{x) < m},G' m :— 

{x E X : f{x) < m}. The following properties hold: 

(i) f and f have the same points of continuity and f = f at such points 

(ii) m — Tessinfx /. 
(Hi) G m F m . 

(iv) F' m C G' m C F' m U Z' m , where Z' m is of first category and contained in an 
Aronszajn null set. In particular, G' m is of first category (contained in an Aronszajn 
null set, contained in a Haar null set) if and only if the same thing is true of F' m . 

Proof, (i) is proved in [371 Theorem 6.4]. 

(ii) Since / is densely continuous, its set of points of discontinuity is of first 
category. Thus, by (i), / = / except on a set of first category. This proves (ii) 

since it is readily checked that modifying a function on a set of first category does 
not affect its topological essential infimum. 

(hi) is trivial when m = — oo since G_oo = F—oo — 0, so assume m > — oo. 
First, F m C G m since / < /. Next, G m C F m , for if x E G m , there is a sequence 

(x n ) tending to x such that \rmf(x n ) = f{x) < m. (Alternatively, notice G m = 

(iv) That F' m C G' m follows from / < /. Suppose now that x E Z' m := G' m \F' m . 

Then, a; is a point of discontinuity of /, for otherwise f(x) = f{x) by (i). This 

already shows that Z m := G' m \F' n is of first category. In addition, by Theorem 14. 21 
the set of points of discontinuity of / in X\F^ -and therefore Z' m - is contained in 
an Aronszajn null set. The "in particular" part is then obvious. □ 

From Lemma 15 - 81 if / is quasiconvex and densely continuous, the size require- 
ments about F m and F' m in Theorem 15.41 or Theorem 15.51 are satisfied if and only 
if they are satisfied by G m and G' m . Therefore, these theorems are applicable to 
/ and / as soon as they are applicable to either of them (if / is not real-valued, 
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first replace / by arctan/). If so, both are Hadamard differentiable on the same 
dense subset of X. In addition, whenever / and / are Hadamard differentiable at 

the same point x, then Df(x) = Df(x). Indeed, by Lemma [5.81 (i). f(x) = f(x), 

so that, since / < /, 

f(x + th)-f(x) ±+ h S_ f ( ^ 

Df(x)h = lim — < lim I(X+ > I(X) = Df(x)h, 

for every h G X, whence Df(x)h — Df(x)h upon changing h into —h. 

When / is replaced by the use hull / of /, there is no close relative of Lemma l5.8[ 



even though / and / coincide away from a set of first category; see |37[ Lemma 5.1], 

where it is also shown that / is finite. In particular, the "optimal" differentiability 

properties of / do not provide any useful information about the differentiability 
properties of /. 

6. Gateaux differentiability on a dense subset 

We now investigate the differentiability properties of densely continuous quasi- 
convex functions without making the additional assumption that F m is nowhere 
dense. Recall that F m is always of first category and may fail to be nowhere dense 
only when m > — oo and dimX = oo. Several special cases when F m is nowhere 
dense were discussed in Section [5j 

If X is a vector space and C c X, we set 

(6.1) C* := U d i 



where Z denotes a vector subspace of X and the closure C D Z is relative to the 
natural euclidean topology of Z. This means that is the sequential closure of 
C in the finest locally convex topology on X ( |41[ p. 56]) or in any finer topology 
(such as the finite topology [23]), although such characterizations will not play a 
role here. Nonetheless, it is informative to point out that, in general, (C*)* ^ C* 
(see a counter example in (H]), so that is not the closure of C for any topology 
on X. 

In this section, the only important (though trivial) features are that C C C$ 
and C C if X is a normed space, that C$ is convex if C is convex, that Y$ = Y 
for every affiiic subspace Y of X, plus the following alternative characterization of 
X\C$ when C is convex. 

Lemma 6.1. Let X be a vector space and C C X be a convex subset. Then, 
x 6 X\C^ if and only if, for every h G X\{0}, there is th > such that (x — 
t h h,x + t h h)r\C = 0. 

Proof. Suppose first that x £ C*, so that x ^ G. It suffices to prove that if h G 
X\{0}, then x + th £ C for t > small enough. Indeed, since x £ C, the same 
property with h replaced by — h yields the existence of th- Now, if x + t n h G C for 
some positive sequence (t n ), it is obvious that x G C H Z with Z :— span{a;, ft}, so 
that x G C* , which is a contradiction. 
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Suppose now that for every h £ X\{0}, there is th > such that (x — thh, x + 
thh) n C = 0. In particular, x ^ C. By contradiction, if a; £ C$, there is a finite 
dimensional subspace Z such that x £ C PI If so, C f~l Z ^ 0, so that the relative 
interior of C n Z is also nonempty. If z is any point of this relative interior, 
then (x,z\ C C D Z ([351 P- 45]). Thus, with h := z - x ^ 0, it follows that 
(.t — th,x + th) n C D (x, x + th) ^ for every t > and a contradiction is 
reached. □ 

It follows from Lemma 16.11 that if C is convex, then x £ C* if and only if there 
are h £ A\{0} and th > such that (x,x + thh] C C. (In particular, when C is 
convex, C* is unchanged if dimZ < 2 in (|6.1|l .) 

From Lemma 16. 1[ it is hardly surprising that should have something to do 
with Gateaux differentiability. The relationship is fully clarified in the proof of the 
next theorem which, in a weaker form, is a generalization, of Theorems 15 .41 and 15 . 51 



Theorem 6.2. Let X be a separable Banach space and let f : X — > K be quasicon- 
vex and densely continuous. With m := Tessinf^ /, suppose that has empty 
interior. Then, f is continuous and Gateaux differentiable on a dense subset of X. 
If X is reflexive and is contained in a Haar null set, then f is continuous and 
Gateaux differentiable on X except at the points of a Haar null set. 

Proof. If F' m is of first category, then F' m is nowhere dense by Theorem 13.21 (i) . 
Thus, F m is nowhere dense and a stronger result follows from Theorems 15.41 and 
15.51 Thus, in the remainder of the proof, we assume that F' m is of second category. 
In particular, m > — oo. 

o o 

Since F' m is of second category, F' m ^ by Theorem [331 (ii) . Let x £ F^ n \F^ 
and h £ X\{0} be given. By Lemma [6.11 there is th > such that (x — thh,x + 
thh) C X\F m . After shrinking th if necessary, we may assume that (x — thh,x + 

t h h) C K\Fl. Since F^\Fl C F^\F m C /^(™), it follows that / = m on 
(x — thh, x + thh), so that the derivative of / at x in the direction h exists and is 
0. Since h £ X\{0} is arbitrary, / is Gateaux differentiable at x with Df(x) = 0. 

o 

Thus, / is Gateaux differentiable at every point of i^\i f1 * l . 

o o 

Now, let x £ F' m be given and let r > be such that B(x, r) C F' m . Since F^ has 

o o / o \ 

empty interior, is dense in F' m , so that there is y £ B(x, r)Cl I F^\F^ ) . Let 

e > be such that B(y,e) C B(x,r). Since y £ F^ and F^ is convex, B(y,e)\F? n 
is of second category by Lemma l2~3l whereas the set of points of discontinuity of / 
is only of first category. As a result, B(y,e)\F* l contains a point of continuity z of 
/. Therefore, z £ B(x, r) and / is both continuous and Gateaux differentiable at z. 
From the above, / is continuous and Gateaux differentiable at the points of a 

o o 

dense subset of F' m . By convexity, F' n ^ is dense in F' m , so that the same subset 
is also dense in F' m . On the other hand, since Hadamard differentiability implies 
continuity, / is continuous and Gateaux differentiable at the points of a dense 
subset of X\F' m by Theorem 14.21 Altogether, this proves that / is continuous and 
Gateaux differentiable at the points of a dense subset of X. 

To complete the proof, assume that X is reflexive and that F^ is contained in 
a Haar null set. In particular, F^ has empty interior. Thus, from the above and 
Theorem [121 the set of points where / is not Gateaux differentiable is contained in 
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a Haar null set. Indeed, such points can only be in F' m n F^ (contained in a Haar 

o 

null set), or in dF^ (Aronszajn null since F' m ^ 0) or in X\F! m and contained in 
an Aronszajn null set. 

By Theorem 13. 3\ the set of points of discontinuity of / is also Haar null and so 
the set of points where / is not continuous or not Gateaux differentiable is Haar 
null. □ 

The counter example to Theorem 15.51 given in the previous section also shows 
that the reflexivity of X cannot be omitted in the second part of Theorem 16.21 

Remark 6.1. Theorem \ 6.S\ or any of its corollaries below is a statement about 
the differentiability properties of f on the entire space X. In particular, it does not 
record the fact that a stronger property is true at the points of X\F! m (Theorem \4-S\ 
recall that X\F! m is always semi-open). 

The set F^ is often much smaller than F m . In the next four corollaries, F^ (but 
not necessarily F m ) has empty interior, although this is not explicitly assumed. 

Corollary 6.3. Let X be a separable Banach space and let f : X —¥ K be quasicon- 
vex and densely continuous. With m :— Tessinfx /, suppose that for every finite 
dimensional subspace Z of X, there is a < m such that F m n Z C F a . Then, f is 
continuous and Gateaux differentiable on a dense subset of X. 

If X is reflexive, f is continuous and Gateaux differentiable on X except at the 
points of a Haar null set. 

Proof. By Theorem 16.21 it suffices to show that F^ has empty interior and that it 
is contained in a Haar null set if X is reflexive. 

From (|6.1I) and the assumption F m n Z C F a when dimZ < oo, it follows that 
C U a < m F Q . Since the sets F a are linearly ordered by inclusion and nowhere 
dense when a < m, l J a<m F a — l J a<m . a ^QF a is of first category. Thus, F^ 
has empty interior. In addition, if X is reflexive, then F a is Haar null, so that 
Ua<m,ateQF a is Haar null. □ 

Corollary 6.4. Let X be a separable Banach space and let f : X —¥ K be quasicon- 
vex and densely continuous. With m :— 7~essinfx /, suppose that F m is contained 
in some proper affine subspace of X. Then, f is continuous and Gateaux differen- 
tiable on a dense subset of X. 

If X is reflexive and F rn is contained in some proper Borel affine subspace, f is 
continuous and Gateaux differentiable on X except at the points of a Haar null set. 

Proof. With no loss of generality, assume dimX = oo (if dimX < oo, Crouzeix's 
theorem gives a stronger result). Let Y denote a proper affine subspace of X 
containing F m . Evidently, F^ C Y$ and, as noted at the beginning of this section, 
Y* = Y since Y is affine. Since a proper affine subspace of X has empty interior, 
F^ has empty interior, so that Theorem 16.21 applies. 

Since Y ^ X, there is £ 6 X\{0} such that M£ intersects no translate of Y at 
more than one point. Hence, if E C X is Borel and n(E) :— Ai(M£ n E) where Ai 
is the Lebesgue measure on R£, then fj,(x + Y) = for every x € X provided that 
Y is Borel. Thus, if X is reflexive, Theorem 16.21 ensures that / is continuous and 
Gateaux differentiable on X except at the points of a Haar null set. □ 

An interesting outcome of Corollary 16.41 is: 
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Corollary 6.5. Let X be a separable Banach space and let f : X — > R be even, 
quasiconvex and densely continuous. Then, f is continuous and Gateaux differen- 
tiable on a dense subset of X. 

If X is reflexive, f is continuous and Gateaux differ entiable on X except at the 
points of a Haar null set. 

Proof. If m := 7~essinfx / = — oo, a stronger property is proved in Corollary 14.31 
Accordingly, we only give a proof when m > — oo and F m ^ since Theorems 15.41 
and 15.51 yield a better result when, more generally, F m is nowhere dense. 

Since / is even, F m = —F m and so 6 F m by the convexity of F m ^ 0. As a 
result, the convex cone CF m := Ux>oXF m is the countable union CF m = U n eN nF m 
(of first category) and CF m — —CF m , so that CF m is a vector subspace of X of 
first category. Thus, CF m ^ X. Since F m C CF m , Corollary 16.41 ensures that / is 
continuous and Gateaux diffcrentiablc on a dense subset of X. 

If X is reflexive, the result also follows from Corollary 16.41 if we show that F m 
is contained in a Borel proper (vector) subspace of X. This can be seen by a mod- 
ification of the above argument. Specifically, F a = —F a for every a 6 R, not just 
a = m, whence F a = —F a and CF a := Uj^oAFq = U n gNrjF Q . Thus, CF a is 
a Borel vector subspace of X for every a € R such that F a ^ and CF a = 
otherwise. 

If a < m, then F a is nowhere dense (Theorem I3.1j) . whence CF a is of first 
category. The subspaces CF a are linearly ordered by inclusion and so U a < m CF a = 
U Q <m,aeQC^Q is a Borel vector subspace of X of first category (F m ^ ensures 
that F a ^ for some a < m, so that CF a is a subspace). Obviously, F m — 
U a <mF a C U a<m F a C U Q<m CF Q ,. This completes the proof. □ 

Below, we show that differentiability on a dense subset is still true when, more 
generally, / is invariant under the action of a linear periodic map T with no fixed 
point. By this, we mean that f(Tx) = f(x) where T e jC-(X), T p = I for some 
integer p > 1 and 1 is not an eigenvalue of T, so that Tx = x if and only if x = 0. 
In Corollary EH p = 2 and T = 

Corollary 6.6. Let X be a separable Banach space and let f : X — > R be quasi- 
convex, densely continuous and invariant under the action of a linear periodic map 
T with no fixed point. Then, f is continuous and Gateaux differentiable on a dense 
subset of X. 

If X is reflexive, f is continuous and Gateaux differentiable on X except at the 
points of a Haar null set. 

Proof. We only show how the proof of Corollary 16.51 can be modified to yield the 
desired result. The key point of that proof is that, when m > — oo and F m / 0, 
the cone CF m is both of first category and a vector subspace of X. That CF m is 
of first category is true irrespective of any symmetry. In the proof of Corollary [631 
that CF m is a vector subspace of A is an immediate by-product of F m — —F m , but 
a routine check reveals that this remains true if, for every x € F m , there is e > 
(possibly depending upon x) such that —ex G F m . The proof that this holds under 
the hypotheses of the corollary is given below, with F m replaced by any nonempty 
convex subset E C X invariant under T (i.e., T(E) C E). 

Since the existence of e is obvious if x — 0, assume x ^ 0. With the notation 
introduced before the corollary, it is clear that x,Tx, ...,T p ~ 1 x 6 E. Furthermore, 
since T p = I, the point y := x+Tx-\ \-T p ~ 1 x is a fixed point of T, (i.e., Ty — y) so 
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that y = since T has no other fixed point. As a result, = ^x+^Tx+- ■ -+^T P 1 x 
and hence G C x := conv{x, ...,T p ~ 1 x} C E. 

We claim that C x is a neighborhood of in Y x := spanjx, T p ~ 1 x}. Indeed, the 
linear mapping A on W defined by A := (A , X p -i) G K p ^ AA := TFr^jT^x G 
Y x is surjective and ker A contains where 1 := (I, 1). Since the subspace 

Z := | A G W : £f=o A j = o| docs not contain it follows that A : Z -> F x is 

still surjective. Let U C Z denote the neighborhood of defined by ?7 := {A G Z : 
\Xj\ < p^ 1 }. Since linear surjective maps are open, A(U) is an open neighborhood 
of in Y x . Now, A(Z7) = A(L7+p- 1 l) and, if /j£ [Z+p- 1 !, then n = X + p- 1 l 
with A G U, so that ^ > and EijZ (J,j = £j=o (Aj = 1. Thus, every vector 

in the open neighborhood A({7) of in Y x is a convex combination of x, ...,T p ~ 1 x 
and therefore in C x . This proves the claim. 

Since —x G i^, and every neighborhood of in Y x is absorbing, it follows that 
—ex £ C x C E for some e > 0, as had to be proved. 

The continuity of T implies that F a (convex) is invariant under T for every 
a G K. Thus, from the above, CF a is a vector subspace of X whenever F a ^ 0. As 
a result, the proof of Corollarv l6.5l can also be repeated when X is reflexive. □ 

Corollaries 16 . 51 and 16 . 61 remain true when / is replaced by its lsc hull / which, as is 

easily checked, exhibits the same symmetry as /. Also, if Theorem 16.21 is applicable 
to /, it is applicable to / by Lemma l5~8l (ii) since F m C G m :— {x e X : f(x) < m} 

is obvious from / < /. However, there seems to be no reason why the converse 

should be true (compare with Subsection 15.21) . 

7. An example 

We give an example of a function / satisfying all the assumptions of Corollaries 
16.31 and 16.51 (hence of Corollaries 16.41 and 16.61 as well, because Corollary 16.51 is a 
special case of both). This example was used in [37] to produce a nontrivial densely 
continuous quasiconvex function. It will also show that the differentiability prop- 
erties of / at the points of F' m are generally much weaker when F m is not nowhere 
dense than they are when F m is nowhere dense. 

Example 7.1. Let X be an infinite dimensional separable Banach space and let 
(x n )neti C X be a dense sequence. Set L n := spanjxi, x n }, so that L := U n ^L n 
is a dense subspace of X and L is of first category since dimL„ < oo. After passing 
to a subsequence, we may assume L n £ L n +i without changing L. Now, let (a n ) C 
K be a strictly increasing sequence such that linin^oo a n = 1. Set f{x) = 1 if 
x € X\L, fix) = a\ if x £ L\ and f{x) — a n if x G L n \L n _i, n > 2. 

In Example 17.11 m := Tessinf^ / = 1, F a = X if a > 1, -Fi = L is convex of 
first category but everywhere dense and F[ = X. Also, if ati < a < 1, there is a 
unique n G N such that a n < a < a n+ i and so F a — L n is (closed, convex and) 
nowhere dense. If a < a±, then F a = 0. In particular, / is quasiconvex. 

By Theorem 13. II for because / is lsc; see Remark |7. 1 1 below) . / is densely contin- 
uous. Actually, a direct verification shows that the set of points of discontinuity of 
/ is exactly L. If x ^ L and h G A\{0}, then x+th G L if and only if x+th G L n for 
some n. This can only happen if t ^ and for at most one t. Indeed, if x + th G L ni 
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and x + sh G L„ 2 with s^t and (say) 112 > rii, then x + sh — x — th = (s — t)h £ L n2 , 
so that /i G L n2 . But then, s/i £ L„ 2 and so x 6 L„ 2 C £, which is a contradiction. 

The above property shows that if x ^ L and /i G X \{0}, there is th > such that 
(x — thh, x + thh) C X\L. Since / = 1 on X\L, it follows that / is also Gateaux 
differentiable at every point of X\L (and Df(x) — 0). Note that L has countable 
dimension, so that it is Borel and Haar null regardless of the refiexivity of X. 

Now, it is plain that / is even, so that it satisfies the hypotheses of Corollary 
16.51 To see that it also satisfies the hypotheses of Corollary 16.41 let Z be a finite 
dimensional subspace of X and let x G F\ n Z = L D Z (recall m — 1). Obviously, 
L n Z = U„ e rj(Z„ n Z) and L n D Z is a nondecreasing sequence of subspaces of 
Z. Since dimZ < oo, it follows that L n n Z is independent of n large enough, say 
n > no and so i n Z = L na n Z. Thus, a; 6 i„ and so f(x) < a n „ < 1 = m. 

Remark 7.1. in Example \7. 1\ F' a = X if a > 1, F' a = L n if a% < a < 1, where 
n G N is the unique integer such that a n < a < a„+i and F' a = if a < a%. 
Thus, F' a is always closed, so that f is Isc. Since X and all its finite dimensional 
subspaces are ideally convex, this also shows that f is ideally quasiconvex (Definition 
\3.1\) . but F\ is dense with empty interior, so that F\ is not ideally quasiconvex; see 
the comments before Theorem \3.4\ Thus, f is not strongly ideally quasiconvex. 

o 

In general, when to > — oo and F' n is of second category, then F' m ^ is dense 
in F' m (Theorem 13.21 (ii)). If also F m is nowhere dense, then / = m on the open 

o 

set F^Fm dense in F^, so that / is Frechet differentiable with derivative on an 
open and dense subset of F' m . This was used in the proof of Theorem 15. 31 

In the above example, to = 1 and F[ = X is of second category. Since F\ = L is 
dense in X and / is not Gateaux differentiable at any point of F\ (Theorem 15. l\ . 
f cannot be Frechet differentiable on an open and dense subset of F[ but, since 
it is continuous and Gateaux differentiable on the residual subset / _1 (1) = X\L, 
it could conceivably be Frechet differentiable on this set, or perhaps on a smaller 
but still residual subset. Below, we show not only that this is not the case, but 
even that the set of points of X\L where / is Hadamard differentiable is of first 
category. This establishes that, in general, the differentiability properties at the 
points of / _1 (m) are indeed weaker when F m is not nowhere dense than they are 
when F m is nowhere dense. 

The notation of Example 17.11 is used in the next lemma. Also, d(x, L n ) denotes 
the distance from iglto the subspace L n . 

Lemma 7.1. Let (/3 n ) n eN C (0, oo) be any sequence. The set W :— {x G X : 

d{x, L n ) < j3 n for infinitely many indices n G N} is residual in X. 

Proof. By the continuity of d(-, L n ), the set W n :— {x G X : d(x, L n ) < /?„} is open 
in X for every n G N. In addition, Lk G W n for every n > k, so that Lk C U„>jW n 
for every j, k G N. Thus, L C U n >jW n for every j G N so that U n >jW n is open 
and dense in X irrespective of j G N. As a result, U„>j W n is residual and 

coincides with the set W of the lemma. □ 

Theorem 7.2. The set of points where the function f of Example ] 7. 1\ is Hadamard 
differentiable is of first category in X. 

Proof. If x G X and n G N, let £ n = £ n (x) G L n be such that ||a: — £ n || = d(x, L n ) 
(recall dimL„ < oo). In Lemma ITTTI let (3 n :— (1 — ot n ) 2 (notation of Example 1 7. lj) . 
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so that W := {x G X : \\x — £ n || < (1 — a n ) 2 for infinitely many indices n € N} is 
residual in X. Below, we show that / is not Hadamard diffcrcntiable at any point 
x € W\L. Since / is not even Gateaux differentiable at any point of L = Fi by 
Theorem l5.ll (i). this implies that / is not Hadamard differentiable at any point of 
W, which proves the theorem. 

If x G X\L, then £, n ^ x for every n and f(x) — 1. We already know that / is 
continuous and Gateaux differentiable at x with derivative (the latter also follows 
from Theorem[57l](ii)). To show that / is not Hadamard differentiable at x G W\L, 
it suffices to find sequences (h n ) C X and (t n ) C (0,oo), both tending to 0, such 
that lmw^ i^+l^tdM ^ o. 

Set h n := TT^pmjz and t n := \ \x—£ n \ \ 1/2 . That limbec t n = and lim^oo h n = 
follows from the definition of £ n and the denseness of L in X. On the other hand, 



f(x + t n h n ) - f(x) 



1-/(0 > l-«n 



\\x-u\ i/2 ~ \\x-a\ i/2 ' 

where the inequality follows from £ n € L n and / < a n on L n . Since x £ W, there are 
infinitely many indices n such that ||x — ^H 1 ' 2 < l — a n . Therefore, ^j_~"| 1/2 > 1 

for infinitely many indices n. As a result, /( a +*"| 1 ")~/( :r ) does not tend to and the 
proof is complete. □ 

Although Theorem 17.21 demonstrates that the cases when F m is nowhere dense 
or just of first category are different, it does not rule out that / is Hadamard 
differentiable except at the points of a Haar null set. With special choices of X, L n 
and a n , it can be shown that the set of points where / is not Hadamard differentiable 
is not Aronszajn null, but "not Haar null" has remained elusive. 



8. A PECULIAR CONVEX SUBSET OF t 2 

Corollaries 16.51 and 16.61 raise the question whether symmetry or any extra as- 
sumption is really needed to ensure that a densely continuous quasiconvex function 
is continuous and Gateaux differentiable at the points of a dense subset of the sepa- 
rable Banach space X. By Theorem l6.2l this would be true if in (|6.1[) had empty 
interior whenever C is a convex subset of X of first category. 

In this section, we show that such a property does not hold, even when X = £ 2 , 
by constructing a convex subset C of I 2 of first category such that C* = i 2 (see 
(|6.ip ). Similar subsets can be constructed in any £ p , 1 < p < oo. 

Let P := {(xj) : Xj > Vj G N} C I 2 denote the nonnegative cone, a closed 
convex subset with empty interior. Set u :— {j~ 1 )jer$ G P and, for n G N, define 
C n := —nu + P, an increasing sequence of closed convex subsets (not cones) with 
empty interior. Then, C :— \J n g®C n is a convex subset (even a cone) of £ 2 of first 
category and PcC. 

Lemma 8.1. For every x G ^ 2 \{0}, there is h G -P\{0} such that, for every 
t > 0, xj + thj > for j large enough. 

Proof. Given x G £ 2 , choose a strictly increasing sequence (n,) C N such that 
Sj^wi+i^i < i~ 4 - For j G N, let hj := \xj\ if 1 < j < m and hj := i\xj\ if 
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i 2 K^j^m+i x j) - 
^% x x 2 + S^r 2 < oo. 



It is obvious that h £ P\{0} and, if t > and i > t 1 , then, Xj + thj > provided 



If x £ A{0}, let h £ P\{0} be given by Lemma O so that if t > 0, then 
Xj + thj > for j large enough. Hence, Xj + i/ij + nuj > irrespective of n £ N for 
the same indices j since Uj > for every j. In addition, the finitely many remaining 
terms Xj + thj + nuj can all be made positive if n is chosen large enough. Thus, 
x + th + nu £ P if n is large enough, whence x + th £ C„ C C for every t > 0. 

Since t > above may be arbitrarily small, there is a sequence of spanjx, h}nC 
tending to x, whence x £ C x . Thus, ^ 2 \{0} C C*. Since also £ P C C C C % , the 
proof that = ^ 2 is complete. 

Anecdotally, that = ^ 2 implies that C has another uncommon feature. Since 
it is of first category, C / f 2 and so, by the so-called Stone-Kakutani property, 
C is contained in a convex subset C such that £ 2 \C is also convex and nonempty 
( }41l p. 71]). But C cannot be a half-space E := {x £ £ 2 : l(x) < a} (let alone 
{x € £ 2 : l(x) < a}) for any a£l and any linear form I on £ 2 , continuous or not. 
Indeed, if C C S, then C 1 '" C Since every linear form is continuous on finite 
dimensional subspaces, if follows from (|6.1|) that = S. Since S 7^ £ 2 is obvious, 
a contradiction is reached with C* = ^ 2 . 

The procedure used in Example l7.1l shows how to construct quasiconvex functions 
/ with F rn = C which, in addition, are continuous at every point of X\C (and so, 
densely continuous). Indeed, with the same increasing sequence (a„), it suffices to 
define f(x) = ct\ on C\,f{x) = a n on C n \C n _i,n > 2 and f(x) = 1 on X\C. 
If x ^ C, then x C n for every n and so there is an open ball B(x,s n ) such 
that B(x, e n ) n C„ = 0. Thus, if y G B(x,e n ), then either /(y) = 1 (if y £ C) or 
f{y) > «n+i (if 2/ £ Cj w hh j > n + 1). In either case, /(?/) is arbitrarily close 
to 1 if n is large enough, which proves the continuity of / at x. Unfortunately, 
the Gateaux differentiability question seems difficult to settle at the points x £ C, 
irrespective of the choice of the sequence (a n ). (The characteristic function of X\C 
is nowhere Gateaux differentiable, but since it is also nowhere continuous, it is not 
a counter example.) 

9. Essential Gateaux differentiability on a dense subset 

In the previous section, we saw that the question whether every densely con- 
tinuous quasiconvex function on a separable Banach space X is continuous and 
Gateaux differentiable on a dense subset of X is not fully resolved by Theorem 
16.21 We complete this paper by showing that a positive answer can be given if the 
conditions for differentiability are suitably relaxed. 

The classical concept of Gateaux differentiability of / at a point x requires 
lim t _>o /(a+t/t t ) " /(a) = l x {h) for some l x £ X* and every h £ X\{0}, that is, for 
every h £ Sx, the unit sphere of X. If this holds only for h in a dense subset Q of 
Sx, then l x is uniquely determined by fl, but since different subsets f2 may produce 



that j > rij + 1. 



□ 



20 



PATRICK J. RABIER 



different linear forms l x , this does not provide a sound generalization of Gateaux 
differentiability. On the other hand, if fi is residual in Sx and if V x G X* is similarly 
defined when Q is replaced by another residual set of directions £1', then l' x = l x 
since fi n ft' is residual, and therefore dense, in Sx (since X is a Banach space, Sx 
is a complete metric space and so a Baire space). 

Accordingly, we shall say that / is essentially Gateaux differentiable at x if 
lx{h) — hmt-i-o /( a + f ^)~/( a: ) f or some l x g X* and every h in some residual subset0f2 
of Sx - If so, from the above remarks, the (essential) Gateaux derivative Df(x) := l x 
is well defined and independent of ft. The next theorem states that for densely 
continuous quasiconvex functions on a separable Banach space, continuity plus 
essential Gateaux differentiability always hold on a dense subset. For clarity, we 
expound the simple geometric property of convex sets used in the proof. 

Lemma 9.1. Let X be a real vector space and let C £ X be a convex subset. 
Given x G X\C and h G A\{0}, there is th > such that either {x,x + thh) C C 
or [x,x + t h h) C X\C. 

Proof. Suppose that there is no th > such that (x, x+t^h) C C, i.e., that (a;, x+th) 
contains a point of X\C for every t > 0. We show that there is th > such that 
[x, x + th) C X\C. If not, since x £ C, there is a decreasing sequence (t n ) C (0, oo) 
such that lim£„ = and x + t n h G C for every n. Since C is convex, x + th G C for 
t G [t n , f i] and every n. Thus, (x, x + t%h] C C contains no point of X\C, which is 
a contradiction. □ 

Theorem 9.2. Let X be a separable Banach space and let f : X — > R be qua- 
siconvex and densely continuous. Then, f is continuous and essentially Gateaux 
differentiable on a dense subset of X. 

If X is reflexive, f is continuous and essentially Gateaux differentiable on X except 
at the points of a Haar null set. 

Proof. As in the proof of Theorem 16. 2\ we may assume that m := Tessinfx / > 

o 

— oo and that F' m is of second category, whence F' m ^ by Theorem 13.21 (ii). We 

o 

shall show that if x G F' m \F m , the essential Gateaux derivative of / at x exists and 

o o 

is 0. Since F^ n \F m is residual in F' m and / is continuous at the points of a residual 

o 

subset oiF' m , this implies that / is continuous and essentially Gateaux differentiable 

o 

at the points of a residual subset of F' m and hence at the points of a dense subset 
oiF' m . By Theorem 14.21 and Theorem l3.2l (iii). / is Hadamard differentiable at the 
points of a dense subset of X\F' m , which proves the first part of the theorem. 

o 

Let then x G F' m \F m be given. The set H x := {h G A\{0} : (x,x + t h h) C F m 
for some th > 0} is a con^ (invariant by multiplication by positive scalars). Also, 
H x C X\{0} and, if h G H x , then h G n(F m — x) for any n G N such that n^ 1 <th- 
Since F m is of first category, H x C Linen n(F m — x) is of first category in X and, 
hence, in A\{0} as well (if E is a closed subset of X with empty interior in X, then 
i?\{0} is a closed subset of A\{0} with empty interior in A\{0}). 



^By the corollary of the Kuratowski-Ulam theorem used in the proof of Theorem 19.21 the 
definition is unchanged if it is required that h belongs to a residual cone in X\{0}. 
^It is also convex, but this is unimportant in this proof. 
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That H x C X\{0} is a cone implies that it is homeomorphic to (0, oo) x (H x C\Sx) 
through the homeomorphism z — > (||z||, z||z|| _1 ) ofX\{0} onto (0, oo) xS^. Since 
homeomorphisms preserve Baire category, (0, oo) x (H x C\Sx) is of first category in 
(0, oo) x Sx- By a classical corollary of the Kuratowski-Ulam theorem [33J Theorem 
15.3], it follows that H x n Sx is of first category in Sx- (The use of this theorem 
requires the topology of Sx to have a countable base, which is the case since X is 
separable.) Equivalently, Sx\H x is residual in Sx- 

Now, if h £ Sx\H x , then [x,x + thh) C X\F m for some th > by Lemma 
19.11 Thus, if h £ Sx\(H x U —H x ) (also residual in Sx), there is th > such that 

o 

(x — thh,x + thh) C X\F m . Since x £ F^, it is not restrictive to assume, after 

o 

shrinking th if necessary, that (x — thh,x + thh) C F' n \F m C / (m). This makes 
it obvious that the derivative of / in the direction h exists and is 0. 

Suppose now that X is reflexive. From the above and Theorem 14. 2 [ the points 
where / does not have an essential Gateaux derivative can only be in F m , or in dF^ 

o 

(Aronszajn null since F' m ^ 0) or in a subset of X\F' m contained in an Aronszajn 
null set. Thus, to show that this set is contained in a Haar null set, it suffices to 
check that F m is contained in a Haar null set. This follows from F m C U a <m,aeQFa 
since F a is convex with empty interior when a < m (Theorem l3.1[) and X is reflexive 
(Subsection l2~2j). 

Since the set of points of discontinuity of / is Haar null (Theorem 13. 3|) and the 
union of two Haar null sets is Haar null, the proof is complete. □ 

There is a close connection between Theorem 16.21 and Theorem 19.21 : If H x = 
in the above proof, then / is Gateaux diffcrcntiable at x (and Df(x) — 0). On the 
other hand, by Lemma T6. II . H x = if and only if x £ F^. 
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